Adobe’s new Premiere Pro AI tools can create videos for you

Adobe is expanding its generative AI tools to Premiere Pro, offering users new ways to enhance your video projects by creating content for you automatically. The new AI capabilities will be powered by Adobe’s new Firefly Video Model, building off of the existing models used in Adobe’s other creativity apps, like Photoshop.

There are three primary features that rely on Adobes generative AI coming your way if you have Premiere Pro: Object removal and addition, generative extend, and generative b-roll.

Related

Sora by OpenAI can make AI-generated videos from any prompt

Sora by OpenAI is a new model that can make AI-generated videos from any prompt, though you can’t use it yet.

Source: Adobe

Let’s break down what’s going to be possible when generative AI lands in Photoshop later this year. First off, object removal and edition is fairly simple. This tool will allow you to simply select an object or are in the video and have it removed or replaced with something else. For example, you might want to remove some copyrighted imagery from a video if you don’t have the license for it. Conversely, you can add things that aren’t there, such as by filling an empty suitcase with money.

Generative extend is useful if you want to hold a particular shot for longer than you did in the original footage. For example, if you have the camera focused on a person but the shot ends too abruptly, you can use Adobe Firefly to extend that clip in a way that still feels natural, rather than just having a still frame for two seconds.

Finally, you can simply generate completely new video footage. Adobe refers to it as generative b-roll, though you can just generate any kind of video. The idea is that you’ll use it for the background of a scene or to have a generic illustration of something. Simply enter the text prompt for the kind of video you want to create, and Premiere Pro will generate the footage for you.

Premiere Pro will support third-party models, too

In addition to powering these features with its own Firefly Video Model, Adobe has also announced its intentions to open up AI integrations for other generative AI models, so you can use whatever fits your needs. Among the examples shown off, we have OpenAI’s Sora, which we already know to be very capable, but also Pika and RunwayAI.

Adobe wasn’t willing to share a timeline for these integrations to be available, though. The current concepts are “early usage explorations”, so we may have to wait a little longer on that front.

Adobe has also committed to ensuring that the source of any content produced using its apps can easily be identified. Through its Content Credentials, Adobe labels AI-generated content so that those viewing it can know whether certain footage is genuine or created by AI. This isn’t new and has been part of Adobe Firefly for a while, though it’s always good to know it also applies to video.